Learning Task-Specific State Representations by Maximizing Slowness and Predictability

The success of reinforcement learning in robotic tasks is highly dependent on the state representation – a mapping from high dimensional sensory observations of the robot to states that can be used for reinforcement learning. Even though many methods have been proposed to learn state representations, it remains an important open problem. Identifying the characteristics existing methods are optimizing to find good state representations, combining them, and adding new characteristics will lead to a more robust method for state representation learning. We define a new characteristic – predictability – and combine it with slowness. We implement these characteristics in a neural network and show that this approach can find good state representations from visual input in simulated robotic tasks.